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SUMMARY 


The Quiet Short-Haul Research Aircraft (QSRA) Project, one of the largest 
aeronautical programs undertaken by NASA to date, achieved a significant cost 
underrun. This is attributed to numerous factors, not the least of which 
were the contractual arrangement and the system of cost and schedule manage- 
ment employed by the contractor. This paper summarizes that system and the 
methods used for cost/performance measurement by the contractor and by the 
NASA project management. Recommendations are made for the use of some of 
these concepts for future programs of a similar nature. 


INTRODUCTION 


The QSRA is a flight research facility to be used by the government to 
develop design criteria and airworthiness standards for quiet, propulsive- 
lift transports. The research application of the aircraft is focused on 
takeoff and landing and other terminal-area operations associated with the 
propulsive-lift mode of flight. The most important priorities for the 
program were safety, cost, and research capability, in that order. Conse- 
quently, the general approach for the aircraft design effort was to "design 
to cost," with emphasis on minimizing absolute cost and/or maximizing 
research capability per dollar, but with no compromises in flight safety. 

In February 1976, the Boeing Commercial Airplane Company was selected 
by NASA to design and build the Quiet Short-Haul Research Aircraft (QSRA) . 

The source selection process included a competitive negotiation between 
Boeing and one other major contractor. The Boeing contract was negotiated 
first. During negotiations substantial reductions were made in engineering 
and manufacturing man-hours. Commensurate work statement reductions were 
made in some, but by no means in all areas. At the same time, Boeing 
proposed a cost-share-award-fee type contract wherein the contractor shared 
$1,200,000, or approximately 6% of the $20 million initial contract cost. 

The contractor was also to receive an award fee, based on performance, which, 
at the discretion of the government, could range from 0 to 5% of government 
target cost. There was no base fee. In addition, the contractor offered to 
share 50% of any overrun, up to a maximum overrun share of $2 million for 
a potential share of $3,200,000. Thus, in addition to the stated NASA 
priority of low cost, there was a strong financial incentive for the 
contractor to remain within and perform under the budget. The Boeing project 



team responded to this challenge with a strong technical and management 
performance which resulted in delivery of the airplane one month ahead of 
schedule and $2.6 million (12%) under target cost, and which earned the 
contractor 100% of the available award fee for each of the six award fee 
periods through aircraft delivery. 

There are many factors, both technical and economic, which contributed 
to the underrun on the QSRA contract. These will be discussed in more detail 
in a subsequent paper. Some may be unique to this program. One factor not 
unique to this program, however, is' the program control method employed. 
Indications of favorable cost performance were apparent early in the program. 
Several design trade-offs resulting in improved producibility were developed 
in the early stages of the program. Also, many engineering and manufacturing 
activities were on or ahead of schedule and on or below budget. This 
performance was reinforced through the NASA project management approach to 
defer contract additions, whenever possible, until funding availability 
within the original budget was assured by realization of the anticipated 
underrun. 

The purpose of this paper is to document the cost and schedule control 
and cost/performance measurement techniques used on the QSRA project in the 
hope that they may prove useful on future programs of a similar nature. 


PROGRAM COST AND SCHEDULE CONTROL 


The objective of a program control system is to provide cost and 
schedule performance visibility in a timely fashion to both contractor and 
government functional managers and to senior management personnel in order 
to ensure project completion within the budget. The system should permit 
early detection of problems and accurate assessment of their impact. It 
should support frequent and up-to-date status reporting and management 
review for decision making purposes. It is highly desirable that the system 
produce a common set of cost reports for review by both contractor and 
government personnel. This facilitates communication and minimizes the 
nonproductive effort that often goes into report generation as well as the 
errors introduced in most cost data manipulations. 

Boeing used established in-house program control systems for the QSRA. 
Existing systems were adapted for work authorization, budgeting, scheduling, 
cost accumulation, performance measurement and reporting. Schedules and 
budgets were established at level 2, 3, and 4 (major activities only) of 
the work breakdown structure (WBS) shown in figure 1. The functional 
manager assigned to each WBS element was responsible for cost and schedule 
performance as well as technical achievement. Work authorizations were used 
to assign manpower and schedules for each job assignment, with charge numbers 
traceable to each WBS element, organization, and cost element. 

The program director controlled the total budget, allocating task 
budgets to each functional manager at 90% of the contract target cost; 
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this established a 10% initial management reserve. The program director then 
used the reserve to adjust budgets for things outside the functional managers' 
control, such as overhead rate changes, internal work statement growth, labor 
rate increases, and schedule changes. 

Boeing employed a "dynamic" budget concept for the QSRA. Contract 
target cost increases for external work statement growth were allocated 
among the appropriate WBS elements and the management reserve. Budget trans- 
fers between the management reserve and other WBS accounts or inter-WBS 
transfers were made for work package refinements or for the reasons discussed 
above. All changes required concurrence of the program director and imme- 
diate notification to NASA of the reasons. 

Although arguments are often made against it there are valid reasons for 
permitting the budget baseline to change on a program like QSRA. A budget 
which reflects design changes, overhead rate changes, etc., is more realistic; 
hence, ,it sustains the motivation and concern of functional managers far 
better than one which is hopelessly obsolete. It also provides a more 
accurate cost picture to project management within the company and the 
government, and facilitates realistic estimates of cost at completion, 
earned value, etc. There are disadvantages as well. A changing budget 
baseline requires considerable effort to maintain. Unless closely controlled, 
it could be used to hide performance problems. For this reason it is some- 
times called a "rubber budget" (ref. 1). Nevertheless, the project manage- 
ment at Boeing and NASA agreed that for a closely knit organization such as 
existed for QSRA, the advantages of rebudgeting outweighed the disadvantages. 
The excellent communication between Boeing and NASA counterparts was a key 
factor in that decision. Also, the general underrun condition early in the 
contract helped considerably. 

Actual costs were accumulated through an automated system with tracking 
by cost element, by WBS, and by organization. This system provided reports 
by WBS of labor costs on a weekly basis, and a monthly comparison of actual 
costs (labor and nonlabor) with budgets. Estimates at completion were 
prepared quarterly by each functional manager to provide the basis for higher 
management action. 

Schedules were established for each WBS element. Significant milestone 
events were identified for activities to which individual budgets were 
assigned. Within engineering, the milestones included significant tests, 
analyses required in support of design, or design release items for which 
release dates mutually acceptable to engineering and manufacturing had been 
established. Within manufacturing, milestones included completion of major 
components or assemblies. 

Cost and schedule information was displayed on a common grid for each 
WBS and posted on the walls of a program control room. The common grid was 
very useful in providing visual cost/schedule "integration." The control 
room also served as the location for biweekly program review meetings, 
where management and other key personnel reviewed cost, schedule, and 
technical performance. 
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CONTRACTOR REPORTING 


The contract specified that monthly financial reports should include 
NASA Form 533M and 533P (ref. 2), or the contractor's equivalent system, for 
each WBS element. NASA Form 533Q reports were required quarterly. The 
level of detail for each WBS element was left to negotiation. 

The computerized cost management system used by Boeing Commercial 
Airplane Co. - Engineering Division for research contracts reported cost by 
WBS element, cost element, and functional organization. It readily lends 
itself to generation of the 533M and 533Q reports. Therefore, these reports 
were submitted for all level 3 WBS elements, and at level 4 for high cost 
items such as wing box, wing trailing edge, and major systems. 

The system does not as easily support the generation of the 533P since 
the basic data elements, planned value of work accomplished and percent 
completion, are not able to be derived from the computerized cost manage- 
ment system without special input and reprogramming. It seemed nonproductive 
to require submission of the 533P at level 3 of the WBS for several reasons. 
First, the estimates of planned value of completed work would be quite 
subjective and therefore of questionable validity. Second, Boeing would not 
rely on this information for cost control, and cost visibility to NASA would 
not be significantly enhanced over that available from the alternate 
financial reports received in lieu of 533P. Finally, it would be costly to 
prepare because the contractor would have to assign both direct and overhead 
personnel to generate level 4 supporting data and reports. Therefore, the 
533P was required only at level 2 of the QSRA WBS. 

The time lag usually present in cost reporting makes problems late in 
surfacing. Since the objective of the government project office is to help 
prevent an overrun rather than just do a good job of tracking one, it was 
judged most useful for NASA to obtain reports which were compatible with the 
Boeing cost control system. These could then be updated in the interim 
period by the resident manager in his weekly reports in order to provide 
near real-time cost monitoring. The expense of preparing the monthly report 
would be only that of duplicating existing data rather than generating new 
data. Most important, the information available from the Boeing internal 
system appeared to meet NASA's needs for cost surveillance very well. 

The monthly reports ultimately provided for each WBS level are shown 
in figure 2. The information which proved to be most useful in tracking 
progress, identifying potential cost or schedule problems, and assessing 
program impact is that listed as "533P equivalent." These reports were a 
combination of graphical and tabular data, which permitted a rapid assess- 
ment of trends as well as absolute cost values at any point in time. If a 
cost growth problem developed, these charts also permitted identification of 
the cause, which might be due to schedule slippages, labor rates, manloading 
variations, material usage, or other factors. Examples of these reports are 
discussed below. 
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Figure 3 illustrates the milestone schedule for the wing trailing edge, 
a level 4 WBS element. Status is shown as of February 28, 1977 (12 mo after 
contract go-ahead) for this and the following figures. The schedule relates 
design, analysis, and manufacturing milestones to the major milestones of 
the program, and identifies specific events which are readily monitored. It 
also shows that the 25% drawing release point was reached 6 wk ahead of 
schedule, while rib fabrication was started 10 wk early. This chart was 
updated biweekly in the Boeing control room and submitted to NASA monthly. 

The engineering release status for the wing trailing edge is shown in 
figure 4. This chart was updated weekly. It compares the actual number of 
releases to the number committed in joint negotiations with manufacturing 
personnel. These consisted of both drawings and advance material releases, 
and it is important that both be on schedule since material must be ordered 
and the manufacturing manload and machine tool usage must be scheduled. The 
chart gives a good indication of the progress of the design effort and also 
suggests that the manufacturing planning activity should be progressing 
satisfactorily, since there were 12 releases ahead of schedule and (equally 
important) none behind at that time. 

Figure 5 measures manufacturing labor progress. It compares cumulative 
actual manufacturing man-hours to the budgeted hours for this WBS element. 

It also shows the "earned" value, which is the estimated, or shop budget 
man-hours for work completed. There was no partial credit earned for work 
started but not yet completed nor for an allocatable portion of the 
management reserve, so the reported earned value was conservative. This is 
very important. Many earned value systems give 50% credit when work is 
started, thus creating an erroneous impression that progress is greater than 
it really is, and hiding cost problems for an extended period of time. 

In addition to the three measures of schedule/functional performance, 
there were two important measures of cost reported for each WBS element. 

One, shown in figure 6, was manloading. Budgeted and actual manpower levels 
in engineering and operations (manufacturing and test support) were reported 
weekly. This report provided a check on cost and schedule performance in 
two ways: 1) early indications of technical problems are often evident in 

manloading deviations from plan; 2) excessive manpower levels, or failure 
to off-load personnel upon task completion, will quickly affect cost 
performance but would be reflected in the weekly manpower reports earlier 
than in the monthly cost reports. In this case, the schedule acceleration 
reported in figures 3, 4, and 5 was achieved even though engineering and 
operations manpower levels were under budget, a happy situation indeed! 

The most important financial data to monitor for each WBS element is 
the comparison of actual and budgeted costs relative to scheduled event 
completion. Figure 7 illustrates that cumulative actual cost for the wing 
trailing edge was modestly below budget although the engineering and manu- 
facturing effort was ahead of schedule. Moreover, the manager's estimate 
at completion (MEAC) was projecting a budget underrun of $82 thousand (8%) 
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through the conclusion of effort on that WBS element. It should be 
remembered that the budget value on this chart is less than the contract 
value by the amount of the management reserve (10%) extracted at contract 
go-ahead by Boeing's program director. By the end of the program the budget 
on this WBSE was underrun by a substantial amount. 

If cost problems had developed, additional visibility was available 
through the variance report shown at the left of figure 7. This itemized 
the cost variance by cost element in order to pinpoint the source of the 
problem. The analysis was made whenever a WBS element exceeded its budget 
by 10%. 


COST/PERFORMANCE MEASUREMENT 


Cost /performance measurement on the QSRA contract was made on a weekly 
basis. The internal cost management system provided weekly output of 
assigned manpower and total labor charges by WBS element. The respective 
control room charts were updated weekly and the data was made available to 
the NASA resident manager for his weekly report to project management at 
Ames. Thus, both the contractor and the government received a weekly 
comparison of actual cost vs budget. Other data received weekly were 
current actual and cumulative average labor rates by functional organization 
(design, technology, manufacturing), additions or reductions in management 
reserve, and a comparison of actual vs scheduled engineering releases. These 
were carefully monitored for signs of trouble. 

In addition to having Boeing cost and schedule reports, the NASA 
resident manager attended weekly staff meetings held by the program manager 
and the design, technology, and manufacturing groups. He also made daily 
visits to all areas of the plant where QSRA work was in progress. This was 
very important for it provided an independent assessment of schedule 
progress vs cost accumulation. The value of this "informal" cost/performance 
measurement can hardly be overemphasized. It lends .credence to the formal 
reports and provides an indication of potential’ problems much earlier than 
does a month-end statement of expenditures. 

Several types of cost/performance measurement were applied by the QSRA 
project office at Ames. The weekly and monthly cost reports were carefully 
checked for the obvious clues to cost and schedule problems. Fortunately, 
there were few of any significance to be found. Occasionally an overbudget 
situation developed in a particular WBS element. Examination of the related 
schedule chart, engineering release performance curve, or manufacturing 
earned value data usually showed that it was because work was being accom- 
plished ahead of schedule. When the work accomplished appeared to be 
consistent with the man-hours expended, no action was taken beyond a 
discussion of the situation with the NASA resident manager or the contractor. 

When the work accomplished was less than the effort expended, or when 
both cost and schedule performance were inadequate, an impact analysis was 
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made. This consisted of an examination of both the budget and schedule for 
that WBS element. If the total budget was small, a 10% or 20% cost overrun 
would not be of great significance to the total program. Since on the QSRA 
program these were always offset by underbudget performance in other areas, 
they were noted for future observation but the "panic button" was not pushed 
It is to be expected that some WBS elements will overrun and some will under 
run on any program. This occurs, in part, because the original budgets are 
established early in the program when there are many unknowns and therefore 
may not accurately reflect the required effort in different areas, and, in 
part, because of the many factors which affect the actual performance. 

The performance discrepancies of greatest concern were. the schedule 
slips in areas affecting other activities or WBS elements. Schedule 
performance, per se, was of the lowest priority on the QSRA program (i.e., 
program cost or aircraft performance and safety was never to be compromised 
to meet a schedule). On the other hand, a delay on one relatively low- 
budget .item (engineering release or manufactured part) can hold up other 
major activities, resulting in a significant reduction in overall produc- 
tivity. Both Boeing and NASA project management were very conscious of this 
and monitored the milestone schedules closely. During the contractor 
functional organizations' weekly meetings, the effects of anticipated and 
actual late completions were evaluated, and "workaround" solutions were 
presented at management status meetings. 

The contractor's management reserve was monitored carefully by the Ames 
project office, since it gives early warning of deviations from plan. Two 
values of management reserve were reported by the contractor, as indicated 
on figure 8. One was based on the internal budget and represented the 
unallocated contract funds held in reserve by the program director. The 
other reserve value was based on the WBS managers' estimates at completion 
for each WBS and represented the outlook at the "working level." Typically, 
the latter value is optimistic since people are either slow to recognize 
or hesitant to admit to problems in their areas of responsibility. On the 
QSRA, however, these estimates were quite realistic, and often conservative. 

The management reserve was sometimes distorted when budgets were 
allocated for work authorized by contract change order but not yet 
definitized. Fluctuations in the mid. 1978 time period reflect this (fig. 8) 
In such cases the reserve was replenished after the contract modifications 
were negotiated. 

Also shown on figure 8 is the hypothetical depletion of the reserve 
which would occur if each WBS activity expended the funding level negotiated 
into the contract. The initial management reserve was created by reducing 
the budget for each WBS element by 10% from the contract value. If the 
cost of each activity had been estimated exactly, each budget would overrun 
by the ratio 10/9 and the reserve would be depleted in accordance with the 
total projected spending rate as shown. While no one expected the reserve 
to be depleted in precisely that fashion, the changing value of the actual 
management reserve, relative to that hypothetical depletion rate, was 
considered by the NASA project management to be an important measure of the 
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financial health of the contract. This was particularly true during the 
early months of the program when design changes, including cost saving 
changes, were occuring and manpower levels were building up. Good cost 
performance reflected by the consistently favorable management reserve during 
the course of the program provided an increasing degree of confidence that a 
cost underrun might be realized. 


CONCLUSIONS AND RECOMMENDATIONS 


The objective of a cost and schedule management system is to help 
project managers to meet program objectives with available resources. To 
meet this objective the system must accurately report actual expenditures 
and accomplishments in each WBS element. It should provide early detection 
of problems and help support development of cost-effective solutions. It 
should. also foster cost consciousness among functional managers and 
facilitate performance measurement by top management. The suitability of 
competing contractors' cost management systems should be considered in the 
source selection process. 

The system used by Boeing on the QSRA contract met the objectives 
admirably. It was adapted from existing systems of work authorization and 
cost accumulation, and hence its maintenance did not impose a financial 
burden on the contract. Yet the information available on a weekly and 
monthly basis was adequate for cost and schedule surveillance and program 
planning by both contractor and government project management. 

The information provided in lieu of the 533P report was especially 
useful in trend analysis and in measuring cost and schedule performance. 

This type of information may not be adequate for all programs, and may not 
even be available in many instances. Nevertheless, government project 
managers should review alternate financial reports and consider accepting 
reports based on the contractor's internal cost and schedule management 
system whenever practical. 

The informal cost and schedule performance measurement available through 
the NASA resident office proved very effective for the QSRA Project. Use 
of a resident office is recommended for all programs with a large budget 
or significant advance in the state of the art. Information is available 
sooner and is probably more candid than could be expected in formal written 
financial reports. The nature and sources of problems are identified in 
detail, and the preferred solution can be communicated to the contractor 
without delay. 
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Figure 1.— QSRA work breakdown structure. 
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Figure 2.— Monthly financial reporting. 
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Figure 5 Manufacturing earned value - wing trailing edge. 
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Figure 8.— QSRA management reserve. 
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